Goto

Collaborating Authors

 bert within-task pre-training fine-tuning


Natural Language Clustering -- Part 1

#artificialintelligence

Classifying things comes quite natural to us: our books, movies and music all have genres; the things we study are split between different subjects and even the food we eat belongs to different cuisines! In recent years we've been able to develop better and better algorithms to classify text: models like BERT-ITPT-FiT (BERT withIn-Task Pre-Training Fine-Tuning) or XL-NET seem to be reigning champions in this category, at least in the 29 benchmark datasets available on PapersWithCode. In recent years we've been able to develop better and better algorithms to classify text: models like BERT-ITPT-FiT (BERT withIn-Task Pre-Training Fine-Tuning) or XL-NET seem to be reigning champions in this category, at least in the 29 benchmark datasets available on PapersWithCode. But what if we don't know the available categories for the texts we want to analyze? Take for example a corpus of conversations or a collection of books or articles that all belong to different specializations within the same subject: labels aren't always as clear cut as spam / not spam, we may not have any idea of how many or what kind of labels to expect, or normal pre-trained classification methods wouldn't have the in-depth domain knowledge required not to classify them as all the same, while not enough material, time or computational power is available to fine-tune a Transformer model.